A Deflated Version of the Block Conjugate Gradient Algorithm with an Application to Gaussian Process Maximum Likelihood Estimation

Many statistical applications require the solution of a symmetric positive definite covariance matrix, sometimes with a large number of right-hand sides of a statistical independence nature. With preconditioning, the preconditioned matrix has almost all the eigenvalues clustered within a narrow range, except for a few extreme eigenvalues deviating from the range rapidly. We derive a deflated version of the block conjugate gradient algorithm to handle the extreme eigenvalues and the multiple right-hand sides. With an appropriate deflation, the rate of convergence depends on the spread of the clustered eigenvalues but not the extreme ones. Numerical experiments in a Gaussian process maximum likelihood estimation application demonstrate the effectiveness of the proposed solver, pointing to the potential of solving very large scale, real-life data analysis problems.